Deep learning proved to be efficient in the learning of models of dynamical system, which is important in engineering for the development of digital twins with correct mechanical behaviour. The depth of networks plays a crucial role in the effectiveness of deep learning. However, the memory requirement for backpropagation scales linearly with the number of layers, which leads to memory bottlenecks during training. Moreover, deep networks are often unable to handle time-series appearing at irregular intervals. These issues can be resolved by considering continuous-depth networks based on the neural ODE. Combination of the neural ODE setting with geometric numerical analysis proved to be efficient in the design of learning algorithms with theoretical guarantees. For instance, the use of reversible integration methods that allow for variable time-steps permits to resolve the mentioned problems. Reversibility of the method ensures that the memory requirement for training is independent of network depth, while variable time-steps are required for assimilating time-series data on irregular intervals. However, at present, there are no known higher-order reversible methods with this property. High-order methods are especially important when a high level of accuracy in learning is required or when small time-steps are necessary due to large errors in time integration of neural ODEs. In this work, we present an approach for constructing high-order reversible methods that allow adaptive time-stepping. Our numerical tests show both the advantages in computational speed and improved training accuracy of the new networks when applied to the task of learning dynamical systems.